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ABSTRACT 


The general area of research is Maritime Domain Awareness, where we 
will be looking at the ship tracking process in prevention and interdiction 
functions. The objective of this research is to demonstrate that the Knowledge 
Value Added (KVA) and Real Options (RO) methodologies can be used to 
assess the current performance of core Maritime Domain Awareness (MDA) 
processes. This type of approach will help with identification and valuation of 
future options for an MDA process. The results of this research will assist MDA 
managers, and operational leaders, in making portfolio management decisions 
for allocating resources to create the correct support tools for MDA processes 
and support systems. The research will provide a proof of concept test of a set 
of decision support tools to support managers in the MDA ship tracking process. 
We also explored a new methodology for determining value added of 
management. 
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I. INTRODUCTION 


The Department of Defense has become very adept at managing assets 
in both war and peacetime operations including movement of logistical supplies, 
battle groups, personnel and information systems. These assets are tangible 
and the metrics appear to be relatively well defined for measurement of 
performance allowing adjustments to be made that attempt to maximize the 
capabilities in any given operating environment. The intangible asset, 
management, which is required to ensure these important activities are executed 
at the appropriate time and place, is not considered in the metrics that are 
currently utilized. It follows, that questions can be posed, such as, “What is the 
value added by information technology to support Maritime Domain Awareness?” 
And, “How much value does management add to the execution of these tasks 
and what is the Return on Investment (ROI) for management’s involvement?” 

In order to evaluate the effectiveness of management and the 
accompanying ROI, a new approach to measurement must be considered that 
will take into account the value of intangible assets like routine, explicit 
knowledge and management’s unique creative intuitive knowledge and convert 
the results into common units of output across the enterprise to enable 
comparability of estimates. The measurement tool will take into account the 
performance of organizational assets including personnel, processes, and 
technology. The approach must also account for intangible assets such as 
knowledge, and intellectual capital. Knowledge value added (KVA) provides a 
methodological framework for measuring the value added of these assets as they 
convert inputs into outputs. With this approach it is possible to describe all 
process outputs in common units. Using the traditional valuation technique of 
“market comparables,” it is possible to assign a price per unit of output that 
enables the use of common financial metrics such as discounted cash flow, net 
present value, ROI. Further, the KVA framework can be used to calibrate 

management’s creative outputs in terms of the amount of time it takes an 

1 



average manager to learn how to make these kinds of decisions through formal 
and informal education, training, and experience. 

Using this technique, a value can be determined for intangible assets such 
as management’s creative activities that have been labeled “management dark 
matter” (Housel and Kanevsky, 2007). Further, this new source of data can be 
used, not only to conduct analysis on the value of management, but may also be 
used to structure options for future strategic planning within an organization. 

The measurement framework selected was developed at the Naval 
Postgraduate School (NPS) by Dr. Thomas Housel and Dr. Jonathan Mun. The 
approach utilizes the Knowledge Value Added and Real Options (KVA+RO) 
valuation framework and addresses the difficulty in measuring intangible assets 
such as management. The KVA+RO analysis was originally designed to 
measure IT portfolio acquisitions and allow for the development of real options 
utilizing Monte Carlo simulation and portfolio optimization techniques to estimate 
the risk-return tradeoffs. This valuation allows decision makers to look at the 
options available to them to maximize return on investment while ensuring 
optimization of resources and risk mitigation. 

This thesis extends original work by Housel, Bell, and Kanevsky on the 

KVA framework by demonstrating that it can be adapted for use in the valuation 

of management in the same manner it is utilized to value other productive assets 

within processes. The research will begin with a discussion of current 

management valuation reports and how they fail to provide defensible estimates 

of the value added by individual managers in a comparable, objective way that 

can be effectively utilized by decision makers. The KVA+RO valuation 

framework will be discussed in detail in the Second section to demonstrate how 

this framework can be used to place a value on all productive assets. The Third 

section will be a demonstration of the extension of the KVA methodology to 

measure the value added by management using a case study of the Maritime 

Domain Awareness (MDA) Track Generation process. The case study provides 

a proof of concept of the use of a new measure of management’s value added 

2 



within the context of the existing KVA approach and will demonstrate the ability to 
value productive assets, including managers, and utilize that valuation to 
recognize the options available to decision makers. This section will also 
demonstrate how the KVA data can be used to inform the analysis of real 
options. Section Four will be a discussion of the results from the proof of concept 
and how the results can be utilized to provide a metric for management and 
empower decision makers with options steering away from subjective analysis of 
management performance and replacing it with a defensible and objectively 
derived quantitative value. Section five will discuss utilizing the KVA approach to 
measure the value added by management within the context of the MDA proof of 
concept. The discussion will cover how valuation of management can be 
approached using the KVA methodology and how decision makers can be more 
effective at understanding the value of management within an organization. 
Section Six will discuss recommendations for implementation of this technique, 
limitations and future research on the subject matter. 


3 
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II. PRESENT DAY IDEAS OF VALUATION 


Industry leaders recognize a need to show the value of management but 
most of the research done focuses on whether or not management has an effect 
on the overall organizational value or how the value of an organization can be 
increased with management. The research does not look at the effect of 
individual managers on an organization or objective metrics to measure the value 
added by those individual managers. The literature reviewed will be briefly 
discussed to further demonstrate the need for a metric allowing us to perform a 
valuation of managers enabling decision makers to see how much value a 
manager adds to an organization. 

“Does Corporate Governance Effect Firm Value?” was the question 
posed by Black, Jang, and Kim (Black, et al., 2003). They answered this question 
by developing a corporate index of 526 Korean companies and conducting 
statistical analysis to find a correlation between corporate governance and firm 
value. A key factor in this research is “that the index is based on information 
obtained on shareholder rights, board of directors in general, outside directors, 
audit committee and internal auditor disclosure to investors, and ownership 
parity” (Black, et al., 2003, p. 3). It is important to note the index used in this 
research because it demonstrates that the manager as a single entity is being 
overlooked when calculating the effect management governance has on an 
organization’s value. The decision maker will only know if the governance of the 
management team as a whole had an effect on a firm’s value and not if the 
management team or a particular manager had a significant contribution to the 
organization’s value. The results of the analysis demonstrated a strong positive 
correlation between corporate governance and firm value which emphasizes that 
decision makers need to be able to apply a metric to the management team and 
individual managers so they will be able to further exploit an organizations assets 
and potentially further increase the organizational value. 


5 



Beiner, Stefan, Wolfgang Drobetz, Markus Schmid, and Heinz 
Zimmerman conducted similar research on corporate governance and the 
valuation of specific firms which also demonstrated a positive relationship. The 
research did not isolate any one manager or management function. The 
researchers used an index constructed of Swiss firms based on a broad 
corporate governance index and applied five additional variables related to 
ownership structure, board characteristics, and leverage to provide a 
comprehensive description of firm-level corporate governance (Beiner, et al., 
2006). The research results show that corporate governance does indeed have 
a distinguishable effect on firm value, but does not allow a decision maker to 
distinguish whether the management team as a whole or individual manager 
contributed to the additional firm value. The decision maker is forced to assume 
that the team as a whole contributed to the increased firm value and in actuality it 
may only be a small portion of the management team’s contributions that are 
causing the value to increase. 

A slightly different approach was used in Klein, Peter, Shapiro, and 
Young’s attempt to assess a correlation between corporate governance and firm 
value with an emphasis on family ownership of a company. The companies used 
in the research were Canadian, but in many cases were traded on a United 
States Stock Exchange as well. When looking for variables to conduct the 
analysis the authors noted the difficulty in finding good “instruments” to work with 
for assessing the correlation of firm value based on corporate governance (Klein, 
et al., 2005). The two variables that were selected were whether or not the 
company was traded in a United States exchange and how long the company 
had been traded on the Toronto Stock Exchange. The results of the research did 
not demonstrate a positive correlation as with the previous two studies and in fact 
the researchers found that corporate governance did not seem to be a factor for 
the companies in Canada. The data would be difficult information for a decision 
maker within a corporation to comprehend because it would not allow for a 
reasonable assessment of the effectiveness of the firm’s management team and 
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how to address either increasing or decreasing organizational value. The 
question might arise: Is a management team required for the organization if this 
study was indeed accurate? The only way to truly know the effectiveness of a 
management team or manager is to independently find a way to measure that 
intangible entity separating it from other factors such as corporate governance. 

Hsiu Ting conducted research on corporate governance in Taiwan utilizing 
the factor of poor economic conditions. The first thing Ting did was acknowledge 
that some previous studies found that corporate governance did not have a factor 
on firm value and then conducted his own analysis to confirm whether or not 
there was a correlation. Ting conducted statistical analysis of 207 companies 
utilizing a simple summary factor and concluded that there was indeed a 
correlation between corporate governance and firm value. The research further 
found that “firms with poor corporate governance mechanism tend to perform 
badly when business cycle goes downward” (Ting, 2006). The most important 
find related to this thesis research is that Ting also found that “the recognition of 
supervisor is an important factor for corporate governance effect as well” (Ting, 
2006). This factor is important because it emphasizes the need for a metric that 
can be used to assess a manager’s performance and the assumption can be 
made that the performance does contribute directly to the value of a firm. 

Michael Armstrong has written a guide for managers to utilize called A 
Handbook of Management Techniques. The guide specifically talks about value 
adding skills and technique for managers and breaks them up into different 
management specializations. The driving force through the guide is to improve 
the manager through better decision making, various methodologies leading to 
improved effectiveness. The dilemma is how does a decision maker assess how 
much added value to the organization will the manager be or has become without 
the metric to measure his or her performance. 1 

1 Thomas Housel and Kanevsky, Valery, “Measuring the Value Added of Management: A 
Knowledge Value Added Approach”, Working Paper, Not Yet Published, 2007 
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This thesis encompasses key components in the development of a 
methodology to measure Management Value Added (MVA) and begins with the 
Knowledge Value Added (KVA) methodology described within the book 
Measuring and Managing Knowledge by Dr. Thomas Housel and Dr. Arthur Bell 
(2001). The literature discusses the shift in paradigm as we enter the information 
age and the importance of knowledge being managed as an asset even though it 
is considered an intangible asset. An important concept that is stressed in this 
literature is the assumption that there is a direct relationship between knowledge 
and the value it creates (Housel and Bell, 2001). This concept allows the current 
thesis research to demonstrate the idea that knowledge does equate to value 
and that if management implicit knowledge can be described then it can be 
measured and translated into a value that can be associated with the cumulative 
value produced by an organization. The Knowledge Value Added (KVA) 
approach was used to conduct a case study as part of the research for this 
thesis. This approach is explained in the literature as a methodology to estimate 
the value of knowledge deployed throughout a company’s core processes 
(Housel and Bell, 2001). The value added is calculated by utilizing a “return ratio 
with the numerator of the ratio being the percentage of the revenue or sales 
dollar allocated to the amount of knowledge required to obtain the outputs of a 
given process in proportion to the total amount of knowledge required to 
generate the corporation’s salable outputs” (Housel and Bell, 2001, p. 40). The 
denominator of the ratio is the cost to execute the process knowledge. 

The next piece of key literature supporting this research is Real Options 
Analysis by Dr. Johnathan Mun. Real Options theory allows an organization to 
make decisions looking at all options available to them. “Traditional discounted 
cash flow approaches assumes a single decision pathway with fixed outcomes, 
and all decisions are made in the beginning without the ability to change and 
develop over time” (Mun, 2006, p. 92). The theory utilizes the value we are able 
to apply to knowledge through Knowledge Value Added methodology and 
conduct Monte Carlo simulation which ultimately provides decision makers with 
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multiple options for an organization. The options may be simplistic in nature or 
give the organization numerous options allowing the organization to execute 
projects at the most advantageous time to gain maximum benefit and value. 
Real Options theory is the second component to the KVA+RO framework utilized 
in the case study to demonstrate the ability for one to place value on an 
intangible asset such as knowledge and utilizes this asset in decision making to 
increase the value of an organization. 

The next step is to lay the groundwork for utilizing a similar approach to 
value management implicit knowledge as an intangible asset and develop a 
metric to consistently measure that value against the organizational value to 
determine how much contribution can be associated with the manager and/or 
management. The cornerstone piece of literature for this thesis is a white paper 
titled Measuring the Value Added of Management: A Knowledge Value Added 
Approach by Dr. Thomas Housel and Dr. Valery Kanevsky (2007). The working 
paper looks at the increasing use of an open business model used in 
organizations today. The model attempts to address the increasing burden on 
management based on the amount of complexity managers must deal with by 
attempting to mitigate risks, resolve uncertainties, improve predictions, and 
exercise the control and oversight necessary to be successful (Housel and 
Kanevsky, 2007). 

In relation to this thesis work, this aspect is important as we address the 
value of managers and how decision makers can become empowered with a 
metric to see how well a manager is performing in the increasingly complex 
environment and be able to correlate that to the overall value of the organization. 
“The idea of management dark matter is introduced in this literature as the use of 
manager’s creative insights when they attempt to predict the future, create 
potential pathways to accomplish the predictions, and control for future risks. 
Those activities that are uniquely associated with management involve the 
creative use of decision heuristics based on their implicit knowledge accumulated 
over years of experience, training and education” (Housel and Kanevsky, 2007, 
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p. 5). This concept is important to understand as we help to further this 
methodology with the case study and explanation of how the Management Value 
Added approach can be applied. 
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III. OVERVIEW: KVA+RO FRAMEWORK 


“KVA measures the value provided by human capital assets and IT assets 
by analyzing an organization, process or function at the process-level” (Housel 
and Mun, 2007, p. 7). It provides insights into each dollar of IT investment by 
monetizing the outputs of all assets, including intangible knowledge assets. An 
output is defined as the end result of an organization’s operations; it can be a 
product or service as shown in Figure 1. 



Figure 1. Measuring Output (From: Uchytil, 2006) 


KVA is designed to assist organizations such as the Department of 
Defense manage IT investments and risk mitigation. The framework’s key 
components are data collection, KVA methodology, and Real Options analysis 
that collectively provide the basis for performance data and analysis on individual 
projects, programs and processes within a portfolio of IT investments (Housel 
and Bell, 2001). There are three methods that KVA analysis can be conducted, 
as shown in the Table 1. 
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Table 1. Three Approaches to KVA (From: Housel and Bell, 2001) 


Steps 

Learning Time 

Process Description 

Binary Query Method 

1 


Identify core process and its sub processes. 


2 

Establish common units 
to measure learning 
time 

Describe products in terms of instructions required to 
reproduce them and select unit of process 
description. 

Create set of binary yes/no questions such 
that all possible outputs are represented as 
sequence of yes/no answers. 

3 

Calculate learning time 
to execute each sub 
process. 

Calculate number of process instructions pertaining to 
each sub process. 

Calculate length of sequence of yes/no 
answers for each sub process. 

4 


Designate sampling period long enough to capture 
representative sample of core process's final 
product/service output. 


5 

Multiply learning time for 
each sub process by 
number of times sub 
process executes 
during sample period. 

Multiply number of process instructions used to 
describe each sub process by number of times sub 
process executes during sample period. 

Multiply length of yes/no string for each sub 
process by number of times this sub 
process executes during sample period. 

6 


Allocate revenue to sub processes in proportion to 
quantities generated by Step 5 and calculate costs for 
each sub process. 


7 


Calculate ROK and interpret results. 



A. DATA COLLECTION: KVA METHODOLOGY 


The KVA methodology was used to fulfill the requirements for this thesis 
as the research project progressed and Maritime Domain Awareness (MDA) 
Track Generation processes unfolded over a period of approximately eleven 
months. The initial phase began with Knowledge Value Added (KVA) data 
collection obtained through interviews with MDA management and MDA process 
Subject Matter Experts (SMEs). Observations were made to obtain average 
learning-time estimates and the number of roughly equivalent MDA process 
instructions required to complete each sub-process. There was an evaluation of 
the framework for measuring the value of cost and benefit of each system within 
each Maritime Domain Awareness (MDA) process and sub-processes required to 
produce an output. Once MDA process data was accurately documented, it was 
supplemented by additional research to compare cost of each MDA process and 
revenue data to establish baseline information. KVA methodology was then 
applied to uncover the value of Maritime Domain Awareness (MDA) Track 
Generation and historical costs for each process. Cost per unit of output was 
calculated by KVA in conjunction with price per unit estimates (based on a 
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market comparables analysis) that provided raw data required for ROI analysis 
(Housel and Bell, 2001). 

B. GAUSSSOFT KVA SOFTWARE 

The second phase involved the use of GaussSoft KVA software that was 
utilized to extract an array of KVA data input: Cost of Employees, Revenues, 
Total Learning Time, Information Technology, and Times Fired. A predetermined 
set of Outputs (Track Generation, Track Quality Assurance, Data Cleansing, and 
Data Acquisition), were used to make strategic, and operational decisions in 
analyzing the economic performance and cost-benefit relation associated with 
the Maritime Domain Track Generation operational processes. 

In Figure 2, there is a profitability model created with GaussSoft KVA 
software for the Maritime Domain Awareness (MDA) process. The resources 
contained the knowledge to produce outputs. 



Figure 2. Profitability Model 

Time to Learn describes the amount of time it takes an average person to 
learn a process. The Time to Learn Model indicates 13.33 weeks for Track 
Generation, 3.33 weeks for Track Quality Assurance, 2.22 weeks for Data 
Cleansing, and 1.11 weeks for Data Acquisition, as shown in Figure 3. So an 
average MDA watch stander would take longer to learn the Track Generation 
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portion of the MDA process in order to execute it properly. It takes less time to 
learn Data Acquisition. 


Time to Learn 



Figure 3. AS-IS Time to Learn Model 


The KVA model indicates a Track Generation of 4.47m, Data Cleansing of 
828.8k, Track Quality Assurance of 372.96k, and Data Acquisition of 4.24k, as 
shown in Figure 4. Even though Track Generation takes a longer time to learn, it 
returns a higher value of knowledge using KVA methodology, but a lower return 
on Data Acquisition. 


KVA 



Figure 4. KVA Model 


14 














Market Comparables is the valuation approach that can be used to 
estimate price per common unit, based on market comparable revenue for 
process outputs. The Market Comparable model indicates a Track Generation of 
$69.7k, Data Cleansing of $5.85k, Track Quality Assurance of $-2.1 k, and Data 
Acquisition of $-8.5k, as shown in Figure 5. Track Generation and Data 
Cleansing shows the most value in the Maritime Domain Awareness 
organization. Track Quality Assurance and Data Acquisition take a substantial 
loss of MDA process outputs. 


Market Comp 


MARKET COMP 

69.70k *69.70T 

60k 
50k 
40k 
30k 
20k 
10k 




-8..5k 


TG.Track Generation TQA,TrackQA 

DC, Data Cleansing DA. Data Acquisition 


Tasks 

2007/01/24 

11:03:51 


Figure 5. Market Comparable Model 


C. REAL OPTIONS 

Real Options analysis incorporates strategic planning, risk assessment 
and management, and investment analysis. Strategic planning affords decision¬ 
makers the ability to leverage uncertainty, limit risk, and generate numerous 
options to increase the value of a project while managing the risk mitigation 
(Mun, 2006). As a financial valuation tool, Real Options allows organizations like 
the Department of Defense to adapt to decisions of response to unexpected 
environmental or market developments (Mun, 2006). 

Figure 5 shows Dr. Johnathan Mun’s integrated risk analysis process up 
close into eight simple steps: 
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1. Qualitative management screening. 

2. Time-series and regression forecasting. 

3. Base case KVA and net present value analysis. 

4. Monte Carlo simulation. 

5. Real options problem framing. 

6. Real options modeling and analysis. 

7. Portfolio and resource optimization. 

8. Reporting and update analysis 


w 

a 


1. List of the 
strategies to 
evaluate 


A 

B 

C 

D 


« Sta rt with a 11st of p otenti a I 
05 p roje cts o r strateg ies to b e 
reviewed and courses of 
actions (COA) to be 
evaluated... 


2. Base case projections for 
each project 


o 

P 


Data Gathering 
Forecasting 




... project prospective 
conditions with the 
assistance of time-series 
forecasting, comparable 
data, historical data, and 
simulated data... 


3. Develop static financial 
models with KVA data 



... generate a series of base case 
KVA financial models for each 
strategic COA based on the 
workflow processes involved... 


4. Risk-based Monte 
Carlo simulation 

Simulation 


>* 

Lognoimdl 



...apply sensitivity and scenario 
analysis coupled with Monte Carlo 
simulation to the analysis to quantify 
the risk and uncertainties involved in 
each element of the strategies... 


5. Framing Real Options 6 ' °P tlons analysis and COA 
quantification 

O Simulation Lattice 

1 \d£- <# 

Closed-Form Models 

2 — = t)+or-/6t 

.. .the relevant COA strategies real °P tions analytics are calculated 
are framed in a strategic real through binomial lattices and closed- 
o pti ons a na lys is. fo rm pa rti a l-d iffe renti al mo de Is with 




.. .stochastic optimization is the next 
(optional) step if multiple projects exist that 
requires efficient asset allocation given 
some budgetary constraints... useful for 
strategic portfolio management... 


8. Reports presentation 
and update analysis 



create reports, make 
decisions, and do it all again 
iteratively overtime... 


Figure 6. Integrated Risk Analysis Process (From: Housel and Mun, 2007) 

In the ensuing stages of the framework, Real Options in the case of this 
thesis examined the Maritime Domain Awareness (MDA) potential opportunity for 
an investment which identified the value of the MDA process options through 
ongoing investments incorporating the outcomes and flexibility in decisions. Dr. 
Johnathan Mun’s Real Options software was utilized using the Lattice and Monte 
Carlo simulation portion. The MDA process was broken down into four options: 

Strategy A: AS-IS option was used to evaluate the current system in 
place. 

Strategy B: Implementing the TO-BE all at once for immediate 
development. 
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Strategy C: Implementing the MDA process in two different stages from 
both the East and West Coast. 

Strategy D: Implementing the TO-BE Radical Development. 

There was a set of inputs utilized for the above options: PV asset, Cost to 
Execute, Timing and Volatility. We used a discount rate of 5% and present value 
of asset of 20 years of revenue. Cost to Execute was based on a scalar factor of 
how much the revenue would be. Timing was based on the assumption that the 
system will last 20 years. Volatility was based on simulation of various average 
learning times and the standard deviation divided by the mean value. By 
applying the Monte Carlo simulation to these options, it allowed change to all 
major inputs in an efficient manner within this model, and the ability to identify, 
quantify, and analyze risk mitigation. The resulting values were compared to 
various MDA scenarios to assess potential improvements and how it could be 
applied in a broader sense to management, and whether or not it brings added 
value to any given Maritime Domain Awareness process. In Chapter IV, the 
results will show the proof of concept and the value of KVA+RO analysis. 
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IV. KVA+RO RESULTS 


The KVA analysis of the MDA implementation revealed that the radical 
approach involving the implementation of the developed software and associated 
systems on both East and West Coast Maritime Intelligence Fusion Centers will 
vastly improve the Return on Investment (ROI) for the implementation and 
enable the fusion centers to increase the volume of data they are currently able 
to handle. 

A. SUBSTANTIAL REVENUE GENERATION 


The investment in the development of this is minimal in comparison to the 
potential generated revenues from the implementation of the track generation 
software developed by Naval Research Laboratories. Table 2 below provides 
estimated revenues based on the data collection from the track generation 
system in live operation during both the “AS IS” and single phase TO-BE 
implementation. The potential for the second phase (Pacific) implementation 
increasing the volume handling capacity of the system allows for an exponential 
increase in track generation which correlates to a substantial revenue increase. 


Table 2. KVA Results - Analysis of Revenues (Annualized) 



Process Title 

"AS IS" 

"TO BE" 

"RADICALTO BE" 

“AS IS” & 
“TO BE” 
Revenue 
Differential 

“AS IS” & 
“RADICAL” 
Revenue 
Differential 

1 

Data Acquisition 

$2,713.94 

$10,847,273.98 

$397,161,466.92 

$10,844,560.04 

$397,158,752.98 

2 

Data Cleansing 

$529,548.62 

$2,277,927.54 

$83,403,908.05 

$1,748,378.92 

$82,874,359.43 

3 

Track Generation 

$953,187.52 

$569,481.88 

$20,850,977.01 

($383,705.64) 

$19,897,789.49 

4 

Track QA 

$238,296.88 

$1,016,931.94 

$37,233,887.52 

$778,635.06 

$36,995,590.64 


Totals 

$1,723,746.96 

$14,711,615.34 

$538,650,239.51 

$12,987,868.38 

$536,926,492.55 


B. IMPROVED PROCESS PERFORMANCE 


All of the sub-processes are improved with the implementation of the 
developed information technology but the most important sub-process to note is 
track generation. Figure 7 shows that track generation is significantly increased 
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with the implementation of the new track generation technology with track 
generation increasing by over two hundred times the capability of the AS-IS 
system. The significance of the increased number of tracks is that the amount of 
information available to the intelligence community customer is vastly increased 
allowing for improved situational awareness. 


IJ QO S O P 
12 3 

AS-IS TO-BE RADICAL 

Figure 7. Tracks Generated (per week) 

C. RETURN ON KNOWLEDGE 

The Return on Knowledge (ROK) metric allows for value to be placed on 
the knowledge contained within both the information systems and the personnel 
who are a part of the process and/or sub-processes. The metric allows for the 
conversion of that knowledge into a quantifiable unit which empowers decision 
makers to make informed decisions on processes based on the value which it 
holds. The KVA analysis demonstrates the ROK for the implementation of the 
newly developed technology significantly increases with both the TO-BE and 
Radical implementation. Table 3 provides a break down of the ROK for each 
implementation broken down by each sub-process. 
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Table 3. KVA Results - Analysis on ROK 


Core 

Process 

Process Title 

"AS IS" ROK 

"TO BE" ROK 

"RADICAL" ROK 

1 

Data Acquisition 

1.21% 

4824.52% 

176544.52% 

2 

Data Cleansing 

235.53% 

1013.15% 

36995.35% 

3 

Track Generation 

423.95% 

253.29% 

9173.84% 

4 

Track QA 

105.99% 

452.30% 

16460.42% 


The increase in ROK indicates more efficient firing of the knowledge within 
each process. The large percentages may seem exaggerated but they are a 
reflection of improved knowledge utilization contributed to the new technology 
implementation. 

The application of KVA allowed discounted cash flow estimates that make 
Real Options analysis feasible. The Real Options framework uses a statistical 
analysis risk management approach to estimating the risk-rewards of the four 
options that were being considered for this project. The data was placed in a 
lattice structured after the options being considered were selected. These options 
included: leaving the system in an AS-IS state, immediate development of new 
technology, two stage development of new technology and radical development 
and deployment of the technology. The statistical analysis conducted also 
known as Monte Carlo simulation included five thousand trials to ensure reliable 
results. The key for real options analysis is to find the value for each option and 
enable the decision maker to make informed decisions that increase the potential 
Return on Investment. 

The results indicated that strategy D, radical development, was the best 
strategy to follow of the four options considered. The results account for 
increased volume of contacts based on overall improved system performance 
from the newly implemented technology. The results from the Real Options 
analysis are depicted in Table 4. 
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Table 4. Real Options Analysis 


Options 

NPV 

Total Strategic Value 

Strategy A 

6.3 Million 

6.3 Million 

Strategy B 

121.9 Million 

140.5 Million 

Strategy C 

51.6 Million 

60.1 Million 

Strategy D 

14.9 Billion 

25.0 Billion 
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V. MEASURING MANAGEMENT 


The KVA analysis of the MDA track generation process options provide 
proof of concept that intangible assets can be accounted for within an 
organization. The question remains how we account for the dark matter or 
intuitive (i.e., non-algorithmically definable) heuristics that allow a manager to 
make creative management decisions that predict the future and potentially add 
value to an organization? Dark matter accounts for a portion of the manager’s 
activity and includes “the use of managers’ creative insights when they attempt to 
predict the future, create potential pathways to accomplish the predictions, and 
control for future risks and uncertainties. Those activities that are uniquely 
associated with management involve the creative use of decision heuristics 
based on their implicit knowledge accumulated over years of experience, training 
and education” (Housel and Kanevsky, 2007, p. 3). Figure 8 shows a track 
generation process. 


Raw Geographical 
Compiliation 

Cleansed Raw Geo 
Compilation 

N 

Data 

N 


, > 

Cleansing | 

> 

Generation |— 

k 

(QA) 1/ 


Track Generated 


Internet (Static) 
And 
GCCS 








Manually Purged 

1” Automated Track 
Generation 

1 Manual Track 
Analysis 


Figure 8. Track Generation 

Management Value Added is structured within the methodology of KVA 
and accounts for the dark matter outputs from manager’s creative knowledge and 
allows us to estimate the amount of dark matter output for each manager. 
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The example that will be used to demonstrate this concept is the 
management of the track generation within the Maritime Fusion Intel Centers. 
The watch officer and watch supervisor will generate dark matter outputs 
continuously while managing tracks generation to: 

• mitigate the risk of false information, 

• avoid poor fusion of data points into tracks, 

• identify tracks that require immediate attention 

• notify those that have a need to know and understand the legalities 
of the information being generated. 

To be successful managers of the track generation process, they must 
utilize their dark matter along with their routine management activities to ensure 
timely delivery of a quality intelligence product to customers in an operational 
environment. 

We measured the dark matter outputs that the watch officer utilized by: 

1. First, segmenting their job description into “dark matter required” 
activities (e.g., finding a discrepancy or ambiguity in generated 
tracks based on their experience when they “smell” something 
wrong) and routine management activities (e.g., QA check track 
reports, posting track information to various web sites) 

2. Second, estimate the learning time required to teach a novice 
manager how to perform each routine and dark matter based 
activity 

3. Third, the number of times that the dark matter outputs were 
generated within 100,800 tracks generated (see Tracks Generated 
figure xx) per week 

The results indicated that of the 100800 tracks generated, 6% raised 
concerns based on the managers experience that required further investigation. 
They indicated that these tracks just didn’t “smell right” (i.e., made non- 
navigational sense). Identifying these problematic tracks required managers to 
use their intuition, implicit knowledge to predict a possible false identification of a 
track. This dark matter ability to predict a possible problem with a track is critical 
because it dramatically affects situation awareness of the customers of this 
information in the operational environment. For example, an experience track 
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manager would recognize that certain tracks were false tracks and not a contact 
of interest thus improving the allocation of scarce operational units to investigate 
and monitor actual contacts of interest. 

The amount of learning time required to teach new managers with college 
degrees how to “smell” problematic tracks was six months of on the job training 
and one week of formal training or 25 weeks. This dark matter knowledge fired 
6000 times per week multiplied by 25 units of learning time equals 150,000 units 
of dark matter output. By multiplying the price per unit of outputs, i.e., $.0136, 
times the total dark matter outputs of 150,000 the value of the dark matter activity 
of the managers was estimated to be $2040 per week. This represents the 
estimated value of the track manager’s dark matter. Table 5 shows the results of 
the MVA test concept. 


Table 5. MVA Results - Simple Concept Test 


Learning Time 
(in weeks) 

Dark Matter Fired 

Total Units 

Cost Per Unit 

Estimated Value 
(per week) 

25 

6000 

150,000 

$.0136 

$2040 


This is a test of concept of the approach to estimating the value added by 
manager’s dark matter in the context of this case example. The example 
provides a metric in which decision makers can assess the value added by 
managers to an organization and estimate the potential and historical revenues 
that might be generated by the firing of the manager’s dark matter. This 
information allows decision makers to look at future strategies and improve the 
utilization of management to continue to increase the value added to the 
organization. The example is simple demonstration of the application in this 
breakthrough concept and requires further testing to include the use of the 
correlation method as discussed in Dr. Thomas Housel and Dr. Valery 
Kanevsky’s white paper. The results could potentially be more dramatic in a 
more turbulent environment, e.g. point of the spear battle space. 
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This research calls into the question the accuracy of KVA based estimates 
that did not include dark matter. However, the possibility exists that all the 
outputs were captured in the KVA based estimates because the dark matter 
outputs are realized in the final product. 
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VI. LIMITATIONS/FUTURE RESEARCH AND 
RECOMMENDATIONS 


A. LIMITATIONS 

There should be more data collected over time for the proof of concept 
methodology and framework providing the value of management in the MDA 
organization, so we could move towards correlation of further testing for 
validation of the MDA management, as well as their supportive operational 
leaders. 

B. FUTURE RESEARCH 

Future Research should include overcoming the limitations by doing more 
research in more turbulent domains and pursue correlation test approach that 
was not utilized in this thesis (see Housel and Kanevsky, 2007). There are future 
projects in the works that include: 

1. KVA+RO framework solution to monitor processes for PEOIWS-1 
beginning with data collection of core processes implementing 
GaussSoft KVA software. The data collection will allow the 
generation of performance parameters that will be used for Real 
Options analysis. 

2. Future Research utilizing KVA+RO framework to assess the value 
of knowledge learned from continuous research utilizing 
FORCENET, which is the future Command and Control system that 
will provide more robust, reliable, and accurate Common 
Operational Picture (COP) to Commander’s making decisions on 
the battlefield. 

3. Management Value Added (MVA) research should continue that 
will evaluate the future of management in organizations such as the 
Department of Defense preventing the risk of poor job performance, 
new product delivery time, and poor acquisition cost estimates. 
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C. RECOMMENDATIONS 

MDA should move to the next step, meaning if you want a continuous ROI 
results, you have to monitor the process overtime and continually compute the 
results against the KVA+RO framework versus sampling only one time as we did 
in this case study. The continuous sampling will enable decision makers to make 
a more informed decision with all options considered against aggregate 
performance feedback. MDA leadership is critical in helping select the options 
for improvement of the MDA process. 

MDA Subject Matter Experts (SME) should utilize capable software such 
as GaussSoft that would provide a more proficient KVA output for continuous 
monitoring of MDA processes in the near future. 
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VII. CONCLUSION 


The proof of concept study demonstrated the research approach can be 
used to value changes through automation to the MDA process of Track 
generation and it helps top level management make portfolio management 
decisions for allocating resources for MDA information systems or reallocation 
based on the ability to exercise the most beneficial option for the organization. 
Track generation automation helps provide the following improvements: 

• faster Track generation of a Maritime Contact of Interest (COI) 

• provide the opportunity to improve fleet Maritime Interdiction 
Operations (MIO) intelligence gathering 

• decrease personnel costs by reducing the number of MDA watch 
standers 

• improve productivity in current MDA processes, allowing more U.S. 
Navy and U.S. Coast Guard ship boardings 

This new Track generation automation will help provide increased value in 
Maritime Domain Awareness (MDA). The technology implementation presents a 
great opportunity for the Department of Homeland Security, U.S. Coast Guard, 
and U.S. Navy to maintain their fight on the Global War on Terrorism when 
conducting Maritime Interdiction Operations (MIO) during peacetime and wartime 
as the threat of terrorism becomes more unpredictable. 
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APPENDIX A. AS IS 


Table 6. KVA AS IS Data 
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APPENDIX B. TO-BE 


Table 7. KVA Actual To-Be Data 
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APPENDIX C. RADICAL TO-BE 


Table 8. KVA Radical To-Be Data 
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